Content-based image retrieval in medical applications: a novel multistep approach

In the past few years, immense improvement was obtained in the field of content-based image retrieval (CBIR). Nevertheless, existing systems still fail when applied to medical image databases. Simple feature-extraction algorithms that operate on the entire image for characterization of color, texture, or shape cannot be related to the descriptive semantics of medical knowledge that is extracted from images by human experts. In this paper, we present a novel multi-step approach, which is specially designed for content-based image retrieval in medical applications (IRMA). In contrast to common approaches, the IRMA-concept is based on a conceptual and algorithmic separation of: (a) image categorization using global features, (b) geometry and contrast registration with respect to prototypes within the categories, (c) extraction of local features, (d) category and query dependent local feature selection, (e) index generation resulting in hierarchical multi-scale blob representations, (f) object identification that links a-priori knowledge on image content to the blobs, and (g) image retrieval processed on the abstract blob-level. The IRMA-concept comprises several benefits when compared to existing CBIR-systems. The image categories enable semantics by prototypes. Furthermore, each image might belong to several categories. A-priori knowledge on both image and query content is adjuncted to indexing. Therefore, the IRMA-concept provides a high amount of content understanding and enables intelligent queries on an abstract level of information. Hence, IRMA promises satisfactory query completion in medical applications.